Black-Box Emotion Detection: On the Variability and Predictive Accuracy of Automated Emotion Detection Algorithms
(A2020-64028)

AUTHORS

Francesc Busquet, University of St Gallen; Christian Hildebrand, University of St. Gallen

KEYWORDS

Emotion detection; discrete emotions; machine learning

ABSTRACT

The ubiquitous availability of image data, advances in cloud-computing, and recent developments in classification algorithms gave rise to a new class of automated emotion detection systems which claim to perform accurate emotion detection from faces at scale. In this research, we provide a tightly controlled validation study using pretrained emotion detection algorithms of the Google ML, Microsoft Cognitive Service, GfK EmoScan, and other platforms to test the robustness and consistency across and within current emotion detection systems. Our results demonstrate considerable variability in predictive validity across emotion detection systems, high variability across different types of discrete emotions with strong positive emotions (such as an open mouth smile) being easier to classify compared to negative emotions such as anger or fear, and we detect sizable positive correlations of theoretically opposite emotions (such as surprise and fear). We provide two modeling strategies to improve prediction accuracy by either combining feature sets or by averaging across emotion detection systems using ensemble methods.